This application relates to outlier detection for A/B testing on online products.
Online controlled experimentation, such as A/B testing for online products, can include two variants that are the control and test conditions. Online controlled experimentation has become common to estimate impact of product change on user engagement and revenue. It has been widely used to guide product development and support decision making for companies providing online products and content. Many decisions regarding features of online products can rely on the results of these experiments.
One of many issues in outlier detection is being able to detect and differentiate outlier values from extreme yet valid observations. In using parametric distribution models there can be issues, considering online experiment data does not usually fit well into a normal distribution. Therefore, described herein are several solutions for these aforementioned technical problems in detection and removal of outlier values from online experiment data.